The patent application relates to a multichannel optics image capturing apparatus wherein at least a first channel comprises an at least partial field of view overlap with the second channel.
Multichannel optics image capturing apparatuses that use mapping optics of juxtaposed optical channels instead of an individual lens exist (Fleet et al., Venkartaraman et al., A. Oberdörster et al.). The individual images generated in the channels are captured separately and combined electronically to an overall image. Some multichannel systems operate with overlap, i.e. a region of a field of view is mapped by several channels. This results in a redundancy which can be used for obtaining additional information such as depth maps.
Such a redundancy, however, also reduces the overall resolution of the image capturing apparatus. Thus, in some cases, the viewing directions of the channels are adjusted such that the sampling patterns of two channels are detuned with respect to one another on an object or object part to be mapped. Thus, a second channel does not only contribute further information to the overall image due to its differing standpoint but also by refining the sampling. With respective performance of the multichannel optics and post-processing of the data collected by the channels, this results also in a higher resolution of the overall image reconstructed from the individual images, called super resolution.
Since adjacent channels are arranged laterally offset on the sensor, the same frequently have a parallax such that an object appears at different angles, depending on the distance. Due to this parallax, detuning of the sampling patterns depends on the distance. When two channels are detuned in the infinite ideal, there will still be distances where the sampling patterns of these channels are not out of phase but coincide exactly. In these cases, super resolution no longer occurs.
Since the resolution of the image capturing apparatus depends on the distance with regard to the overall image, an additional problem occurs. In a single-channel digital camera, the resolution of the optics has to be adapted to the sampling rate of the image sensor for preventing subsampling and related aliasing. This takes place either by designing the mapping lenses themselves or by incorporating a so-called anti-aliasing filter also called optical low-pass. When the angular resolution as in the above case depends on the distance, the low-pass effect of the scene would have to adapted, for each image part according to its distance. However, dynamic locally adaptable optical low-pass filters have so far been unrealizable.
The problem of distance-dependent resolution concerns not only the above described arrangement, but also other multichannel optics image capturing apparatuses, specifically those where the viewing direction of all channels is the same (Fleet et al., Venkartaraman et al.). Additionally, it concerns so-called plenoptic cameras combining a conventional objective with multichannel optics for mapping an object (Ng et al. T. Georgiev et al.).
However, in a simplified manner, sampling points and sampling patterns can be referred to in multi-channel optics capturing apparatuses, however, a detector pixel of the multichannel optics capturing apparatuses does not measure the intensity at a discrete point, but in an integrated manner across a solid angle. However, the described problem remains the same, depending on the object distance, integration areas of detector pixels of adjacent channels overlap more or less. The lower the overlap, the higher the resolution and the more robust the image reconstruction of the overall image. Consequently, also from this point of view, the resolution of the reconstructed overall image is also dependent on the distance.
Generally, the described problem has so far been accepted. It occurs in multichannel optics image capturing apparatus operating with resolution enhancement by overlapping fields of view. Currently, these systems are rarely used and in conventional single-channel optics image capturing apparatuses, this problem does not exist.
A possible existing approach is the provision of a camera array whose channels are individually rotated (Koskinen et al.). In that way, irregular sampling can be generated. Here, it can be problematic that arrays of complete cameras, i.e. sensor and optics as a whole, have to be twisted against one another, which impedes the provision of a multichannel optics image capturing apparatus built in a compact manner. Additionally, the disadvantage can arise that image fields that are twisted against one another cannot be placed beside one another without wasting space when a continuous sensor substrate is used for all channels.
Further, Penrose pixels exist having a specific geometry for generating pseudorandom sampling patterns (Ben-Ezra et al.). The same are used for resolution enhancement with single-channel systems. Oversampling can take place via multiple captures and shifting the cameras between captures. For this solution, different variations of rhombical pixels are necessitated. In multichannel optics image capturing apparatuses, such an implementation would necessitate significant effort for redevelopment of the pixels and for wiring the pseudorandom pixel arrangement.
According to the above statements, there is a need to provide multichannel optics image capturing apparatuses eliminating the stated disadvantages. In particular, multichannel optics image capturing apparatuses are to be provided whose resolution is essentially distance-independent. Additionally, compared to existing solutions, the same ought to be produced easier and be built in a more compact manner. By means of multichannel optics image capturing apparatuses improved in that manner, multichannel optics image capturing methods are to enabled by which high-resolution object captures are possible, which are essentially independent of a distance between multichannel optics image capturing apparatus and object.